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What is claimed is: 

1. An AIU structure synthesizer of semantic information from multimedia non-textual 
documents comprising: 

an error corrector for receiving raw AIU's extracted from said non-textual documents 
and for correcting errors in said raw AIU ! s; 

a primitive identifier connected to said error corrector, for assigning semantics to 
outputs of said error corrector; 

a basic synthesizer connected to said primitive identifier, for grouping related 
outputs of said primitive identifier; and, 

an abstract synthesizer connected to said basic synthesizer, for grouping related 
outputs of said basic synthesizer. 

2. An AIU structure synthesizer of semantic information from multimedia documents as 
claimed in claim 1 wherein said error corrector comprises: 

a plurality of state machines each state machine for receiving a pattern specified in a 
rule and for providing an output; 

a composite state machine for unioning together each of said output from each of said 
plurality of state machines; 

an execute state machine for providing pattern fitting by receiving input AIU ! s and a 
composite output from said composite state machine and by providing corrected raw AIU's 
and a syntax error; and, 

a substitute single character for receiving said syntax error and for providing a 
single character substitution to said input AIU's. 

3 . An AIU structure synthesizer of semantic information from multimedia documents as 
claimed in claim 1 wherein said primitive identifier comprises: 

assign means for assigning semantics to information extracted by an AIU extractor; and, 
mapping means for performing a one-to-one mapping between corrected raw AIU's and 
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primitives . 

4. An AIU structure synthesizer of semantic information from multimedia documents as 
claimed in claim 1 wherein said basic synthesizer comprises: 

an input pattern evaluator for receiving a set of primitives at an input; 

a first decision block connected to an output of said input pattern evaluator for 
providing a false output to said input of said input pattern evaluator; 

an input objects grouper connected to a true output of said first decision block; 

a second decision block connected to an output of said input objects grouper; 

an output objects generator connected to a no output of said second decision block for 
providing output basic objects; 

a logical express evaluator connected to a yes output of said second decision block; 

a third decision block connected to an output of said logical expression evaluator for 
providing a false output to said input of said input pattern evaluator; and, 

an output basic objects storer connected to a true output of said third decision block 
for providing a stored output basic object to said input of said input pattern 
evaluator. 

5. An AIU structure synthesizer of semantic information from multimedia documents as 
claimed in claim 1 wherein said abstract synthesizer comprises: 

an input pattern evaluator for receiving basic objects from said basic synthesizer at 
an input; 

a first decision block connected to an output of said input pattern evaluator for 
providing a false output to said input of said input pattern evaluator; 

an input objects grouper connected to a true output of said first decision block; 

a second decision block connected to an output of said input objects grouper; 

an output objects generator connected to a no output of said second decision block for 
providing output abstract objects; 

a logical expression evaluator connected to a yes output of said second decision block; 

a third decision block connected to an output of said logical expression evaluator for 
providing a false output to said input of said input pattern evaluator; and, 

an output abstract objects storer connected to a true output of said third decision 
block for providing a stored output abstract object to said input of said input pattern 
evaluator . 

6. An AIU structure synthesizer of semantic information from multimedia documents as 
claimed in claim 1 wherein: 

said error corrector attempts to detect and correct syntax errors introduced in an AIU 
extraction process; 

said primitive identifier maps corrected raw AIU f s into primitives in an application 
domain; 

said basic synthesizer groups one or more related primitives into basic objects in said 
application domain; and, 

said abstract synthesizer groups one or more related basic objects into abstract 
objects and performs recursively to construct an abstract structure, 

wherein said error corrector, said primitive identifier, said basic synthesizer and 
said abstract synthesizer build up a hierarchical structure of AIU objects. 
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7. An AIU structure synthesizer of semantic information from multimedia documents as 
claimed in claim 1 wherein said error corrector comprises: 

pattern fitting means based on finite-state machines; and, 

syntactical transformation means. 

8. An AIU structure synthesizer of semantic information from multimedia documents as 
claimed in claim 1 wherein said basic synthesizer comprises: 

pattern matching means for evaluating input AIU's to generate matched AIU's matching 
specified input patterns; 

grouping means for generating groups of said matched AIU f s; 

logical expression means for evaluating logical expression on said groups of said 
matched AIU's; and, 

string manipulation means for generating a set of AIU's with specified output patterns. 

9. An automatic synthesizer of semantic information from multimedia non-textual 
documents comprising: 

SGML converters for receiving textual documents and for providing SGML structures to a 
hyper linker; 

AIU extractors for receiving non- textual documents and for providing raw AIU's 
extracted from said non-textual documents; and, 

an AIU structure synthesizer connected to said AIU extractors for receiving said raw 
AIU's and for providing AIU structures to said hyperlinker. 

* 

10. An automatic synthesizer of semantic information from multimedia documents as 
claimed in claim 9 wherein said AIU structure synthesizer comprises: 

w 

an error corrector for receiving said raw AIU's; 

a primitive identifier connected to said error corrector; 

a basic synthesizer connected to said primitive identifier; and, 

an abstract synthesizer connected to said basic synthesizer for providing abstract 
objects to said hyperlinker. 

11. An automatic synthesizer of semantic information from multimedia documents as claim 
in claim 10 wherein said error corrector comprises: 

a plurality of state machines each state machine for receiving a pattern specified in a 
rule and for providing an output; 

a composite state machine for unioning together each of said output from each of said 
plurality of state machines; 

an execute state machine for providing pattern fitting by receiving an input AIU and a 
composite output from said composite state machine and by providing a corrected raw AIU 
and a syntax error; and, 

a substitute single character for receiving said syntax error and for providing a 
single character substitution to said input AIU. 

12. An automatic synthesizer of semantic information from multimedia documents as 
claimed in claim 10 wherein said primitive identifier comprises: 

assign means for assigning semantics to information extracted by said AIU extractors; 
and, 

mapping means for performing a one-to-one mapping between corrected raw AIU's and 
primitives . 
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